Extraction of Phrase-based Concepts in Vulnerability Descriptions through Unsupervised Labeling

Software vulnerabilities, once disclosed, can be documented in vulnerability databases, which have great potential to advance analysis and security research. People describe the key characteristics of software vulnerabilities natural language mixed with domain-specific names concepts. This textual nature poses a significant challenge for automatic knowledge embedded text. Automatic extraction aspects is highly desirable but demands effort manually label data model training. In this paper, we propose unsupervised methods extract important concepts descriptions (TVDs). We focus on six types phrase-based (vulnerability type, vulnerable component, root cause, attacker impact, attack vector) as they are much more difficult than name- or number-based entities (i.e., vendor, product, version). Our approach based observation that same-type phrases, no matter how differ sentence structures phrase expressions, usually share syntactically similar paths parsing trees. Specifically, present source-target neural architecture learns Part-of-Speech (POS) tagging identify token’s functional role within TVDs, where source trained capture common features found TVD corpus, target linguistically malformed words specific domain. evaluation confirms proposed tagger outperforms (4.45%–5.98%) taggers designed notions identifies broad set TVDs contents. Then, observations, two path representations (absolute relative paths) use an auto-encoder encode such syntactic similarities. To address discrete our paths, enhance traditional Variational Auto-encoder (VAE) Gumble-Max trick categorical distribution thus create Categorical VAE (CaVAE). latent space absolute further apply clustering techniques generate clusters effectiveness CaVAE, achieves small (85.85) log-likelihood encoding accuracy (83%–89%) resulting clusters. The accurately from corpus way. Furthermore, these labeled mapped back corresponding phrases original produce labels used train concept models other corpora. work, (concept classification sequence labeling model) demonstrate utility unsupervisedly study shows outperform (3.9%–5.14%) those datasets previous work due consistent boundary typing by method.